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ABSTRACT 

This document outlines requirements for evaluating 
schoolwide projects funded under Title I Chapter 1 of the Elementary 
euid Secondary Education Act, 1975. The following schcxjl-level 
requirements must be met on a school by school basis; (1) demonstrate 
the aggreg{'4te achievement gains of all disadvantaged students in 
grades 2-12; (2) examine student-level achievement gains and modify 
the program accordingly; (3) monitor and assess the attainment of 
desired outcomes at all grade levels; and (4) conduct an annual 
review. District-level requirements include examining the academic 
achievement of students in the regular program and conducting a 
sustained effects study every three years. A sumjiative evaluation 
must be conducted at the end of the third project year comparing the 
achievement gains attained in grades 2-12 by the schoolwide project 
to the achievement gains of either other dia^veuitaged students in 
the district that same year, or to the achievement gains of 
disadvantaged students in the same school for the 3 years priur to 
the initiation of the project. The following materials are appended: 
(1) a guide for determining project yeais; (2) two examples of 
determining "comparable data" for evaluation purposes; (3) two forms 
for defining the basis of the achievement giun comparison; (4) an 
example of the calculation of average achievement gains; and (5) the 
requirements for demonstrating desired outcomes. (FMW) 
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EVALUATING SCHOOLWIDE PROJECTS 
Mset the requiremenfs of all Chapter 1 programs 



School-level requlremGnts on a school by school basis. 

1 . A ggregate gains for all educationally disadvantaged students in grades 2 to 
12 from Spring of the year prior to project implementation to Spring of the 
first project year: 

0 record total reading and total matfi scores in NCEs and compute 
gain or loss 

0 identify children separately who are educationally deprived based on 
reading and math scores. Those below the 50th NCE in Spring of 
the year prior to project implementation are counted. 

0 average the gains of educationally deprived children only. Hint: Sum 
the gains separately from the losses. Subtract the total losses from 
the total gains and divide by the number of scores in both groups. 

2. Look at $MQnt l^y^l g^/f?g: 

0 for last year's students 

0 make program modifications for children who didn't gain 

3. Monitor and assess attainment of desired outcomes at ai! grade levels using 
the criteria established for "substantial progress." Set up school data bases 
for each outcome in your schoolwlde project application. 

4. Conduct annual review of data in 1, ki, and 3 and disseminate the 
information to parents, teachers and others. 

Contribute data to meet district-level requirements 

5. Look at performance in the regular program 

6. Conjuct sustained effects study every three years. 
Carry out special schoolwide project evaluation requirement 

7. Conduct a summative evaluation at the end of the third project year. 
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SUMMATIVE EVALUATION OF SOHOOLWIDE PROJECTS 



At the end of the third project year a comparison must be made between the gains 
attained in Grades 2-12 by the schoolwide project and the gains attained by either: 

0 the other Chapter 1 students in the district that same year ("other schools" 
comparison); or 

0 the Chapter 1 students in the same school for the three years prior to the 
initiation of the schoolwide project ("same school" comparison). 



Most schoolwide projects have chosen the latter comparison. Since this choice requires 
the use of data collected over a six-year period (years -3, -2, -1, and +3) it is important 
during the first project year to assess the feasibility of obtaining quality data on 
comparable groups. If all of the following questions have positive answers, then the 
"same school" compariscn is feasible. 



1. Did all or most grades in the school receive Chapter 1 services in a// three 
years prior to the beginning of the schoolwide project? 



2. Will the same test be given in the third project year as was used for the 
three years prior to the beginning of schoolwide projects? (If not, was a 
different edition of the same test given?) 



3. Are data available on at least half of the children served each year prior to 
implementation of the schoolwide project? 



4. Were these data collected using an annual testing cycle (e.g. spring to 
spring)? if not, can annual drta be retrieved from student records? 



5. Is it feasible in terms of time and resources available you and access to 
records of students no longer in the school to construct a data base for the 
three years prior to schoolwide project implementation? 



ONLY IF THERE ARE SATISFACTORY ANSWERS TO THESE 
QUESTIONS SHOULD THE "SAME SCHOOL" COMPARISON BE 
ATTEMPTED. 
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Guide for Determining Project Years 
by Year in Whicli Schooiwide Project Began 



Schoolwlde project Is 
implemented In 1989-90 
school year. 



Schooiwide project Is 
Implemented in 1990-91 
school year. 



Year -3 



1^-87 



1987-88 



Year -2 
Year -1 
Year +1 



1987- 88 

1988- 89 

1989- ^ 



1988- 89 

1989- 90 

1990- 91 



Year -i-z 



1990-91 



1991-92 



Year +3 



1991-92 



1992-93 



Note: // may be helpful to write the appropriate years in place of Year -3 to Year +3 on 
the attached pages. 
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What are "comparable data" for purposes of 
evaluating a 8cfK)olwfde profc^ 



The two sets of data for comparing the gains of children In a schoolwide project 
in its third year to the gains of a comparison group need to be alike In temns of: 

o the grades of the children included 

o the pretest cutoff for children Included 

o the testing interval used (fall-spring or annual) 

o the reasons for which children were excluded from the dataset 



Example 1i 



A Chapter 1 school served the following children 

Year -3 - Grades 1-6 reading, children up to 30th %ile 
Year -3 - Grades 4-6 math, children up to 25th %ile 
Year -2 - Grades 1-4 reading, children up to 35th %ile 
Year -2 - Grades 4-5 math, children up to 30th %ile 
Year -1 - Same as Year -2 

Now the school has a schoolwide project, and its funding is based on the unduplicated 
count of children below ttiB 50th percentile in either reading or math 



Comparable databases for the comparison can only include: 

o for reading, children in grades 2-4 whose pretest score was at or below the 
30th %ile 

0 for math, children in grades 4-5 whose pretest score was at or below the 
25th %ile. 
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A schoolwide prq'ect formerly served all children below the 40th percentile in 
reading and in math. However, gains from Year -3 and Year -2 were based on fall-spring 
testing, whereas annual testing was used from Year -1 on. Also, data processing 
eliminated any children in Year -1 who were absent more than 18 days or who had ever 
repeated a gi ade. This reduced the percent of students who had data from 72% in Year 
-3 and 70% Year -2 to 18% in Year -1. 



The database needs to be reconstructed from cumulative folders so that 
comparable data are available on as many children as possible: 



0 for Year -3 and Year -2, complete annual gains from the cumulative folders 

0 for Year -1, confipute annual gains for all children served regardless of 
attendance and retention in grade 

0 for Year +3, use annual gains only for children whose pretest score was 
below the 40th percentile 
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SCHOOLWIDE PROJECT COMPARISON 
Deflning the Basis of the Comparison in the Same School 

Reading Math 

Percentile Grades Percentile Grades 



Sept. Year -3 
Sept. Year -2 
Sept. Year -1 



Sept. Year +1 Schoolwide Project Began 
Sept. Year +2 (No Comparison Required.) 
Sept. Year +3 (No Comparison Required.) 



June Year +3 Do third year gains data exceed the average achievement gains of 

comparable educationally deprived children from the three years prior 
to implementation? 
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AN EXAMPLE OF CALCULATION OF THE AVERAGE ACHIEVEMENT GAIN 
FOR COMPARABLE EDUCATIONALLY DEPRIVED CHILDREN 
FOR THE THREE YEARS PRIOR TO PROJECT IMPLEMENTATION 



Step 1: Determine the lowest percentile In reading and math which was used for 
student selection in the three years prior to Implementation. 

Reading; 30*" percentile 
Math: 25*" percentile 



Step 2: For the students up to and including the percentile points identified in Step 
1, multiply the average NCE gain by the number of students serve:- (N) 
during the year. 



Reading Math 





Ave. 




Total 


Ave. 




Total 




NC£ 


N = 


Ave. 


NCE 


N = 


Ave. 




Gain 




Gain 


Gain 




Gain 


Years 


-5 


72 


-360 


-2 


63 


-726 


Year '2 


2 


77 


154 




60 


'360 


Year-1 


0 


80 


000 


1 


50 


50 


TOTAL 




226 


-206 




173 


-436 



Step 3: Sum the total average gains and divide by the numt>er of students. 

Reading: (-206) / (229) = (-.9) 
Mam: (-436) / (173) ^ (-2.52) 



Step 4: 



Compare gains for Year +3 to the average gains from Step 3 above. 



SCHOOLWIDE PROJECT COMPARISON 



DEFINING THE BASIS OF THE COMPARISON 
BETWEEN THE SCHOOL AND THE LEA 



Step 1 : Extract data on students from sdioolwide project up to the level used by the 
LEA for selection of Chapter 1 students throughout the district. For 
example, if the LEA had enough funds to serve only those students who 
scored at or tjelow the 30th percentile in year + 2 and at or below the 27th 
percentile in Year +3, the schoolwide project vyould use data on only those 
students for comparison purposes. 

Step 2: From the LEA database extract data on students in the same grades as 
those served in schoulvynde projects. 

Step 3: Calculate average NCE scores in both reading and math for students 
selected in Steps 1 and 2. 

Step 4: Subtract Year +2 scores from Year +3 scores. 



Step 5: Compare gains calculated for schoolwide project to those of district as a 
whole. 



School 


LEA 


Year +2 . Year +3 


Year +2 Year +3 


*30 *27 

Percentile 


30 27 


2-6 2-6 

Grades 


★*2-6 **2-6 


Ave. NCE 
Reading 




Ave. NCE 
Math 









*Must be the same as LEA. 

**Must be the same as SchoolwMe Project being compared. 
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DESIRED OUTCOMES IN SCHOOLWIDE PROJECTS 

While desired outcomes may be stated for the whole school, their evaluation MUST 
separate data for the educationally deprived and non-deprived children. The 
accountability requirements for all Chapter 1 schools require that the educationally 
deprived students make substantial progress toward meeting the desired outcomes. Both 
the aggregated data for desired outcomes and individual student's progress must be 
inspected. 

The pro grass of pro-K. Kinder^rt^ and Gmcte 1 children should be evaluated 
through desired outcomes, since their achievement data are not aggregated with that of 
children In grades 2 and above. If a schoolwide project does not have one or more 
desired outcomes stated for the academic progress of the younger children, a desired 
outcome or two in this area should be added. 
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